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ABSTRACT 



This paper explores the development of a set of indicators 
devoted exclusively to the school to work transition. The indicators 
described are directed at monitoring a geographical area as a whole, whether 
it is the nation, state, or a local community. Developing a set of indicators 
is linked to making judgments about the impact we expect new systems to have 
on the achievement of economic adulthood in the United States. It is also 
linked to making judgments about the institutional changes need to ensure 
economic maturation. Three sets of indicators are considered: (1) the final 

outcomes aimed for by improving the school to work transition; (2) 
intermediate outcomes that are the early results of changes; and (3) system 
outcomes that represent the specific changes desired in institutional 
behaviors. Final outcomes include changes in employment patterns, reversing 
the relative earnings decline, increasing the perception of well being among 
citizens, and other impacts on economic success. Intermediate outcomes are 
those related to literacy, finding employment, seeking higher education, and 
changing perceptions about new school to work initiatives. System outcomes 
include changes in program opportunities and increased training and 
employment -related opportunities. An indicator system will be a set of 
measures designed to track the putting into place of the elements of a system 
and whole systems. The indicators will indicate change, but do not, in 
themselves, form a program development and implementation plan. An appendix 
presents notes on performance measures. (SLD) 
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INTRODUCTION 



This paper explores the development of a set of indicators devoted exclusively to 
the school to work transition. The indicators described in this paper are directed at 
monitoring a geographical area as a whole: the nation, or a state, or perhaps a commu- 
nity. We begin by focusing on the nation, but we are also concerned with identifying a 
set of feasible state and community indicators. While cost becomes a large factor in 
tracking individual communities, some key indicators are built upon school- and 
employer-level reporting that could permit aggregation at the community level. 

This paper does not directly address performance measures for funded projects. 
That is, it does not tackle measures of what constitutes successful project implementa- 
tion. Such performance measures deal with how well funded projects serve individual 
youth; the indicators in this paper deal with installing a whole system. However, the 
two are obviously related, and thinking through an indicator system will help in think- 
ing through project performance measures. Comments about performance measures 
required by the new school-to-work legislation (House version) are included as an 
Appendix. 

Developing a set of indicators is linked to making judgments about what impact we 
expect new systems to have on the achievement of economic adulthood in the United 
States. Also, it is linked to making judgments about the institutional changes needed to 
ensure economic maturation. Work on an indicator system will flush out underlying 
assumptions about an effective approach to the school-to-work transition, and could 
lead to clearer thinking. It is not possible to set forth an indicator system without 
revealing underlying views about the changes needed in the support/preparation sys- 
tem, and, therefore, this is not simply an exercise in statistical theory and methods. 

Indicators are inevitably driven by a specific prescription for change, except possi- 
bly for the final outcome measures, which could be agreed upon by people with differ- 
ent prescriptions. While prescriptions in the indicator system are my own versions of 
the necessary ingredients, I believe they generally fit the school-to-work legislation 
proposed by the Clinton administration and now being considered by the Congress. 
There may be room for argument over details, however. Some of these details are in 
elements that may go beyond this legislation. One such example would be efforts to 
increase industry investment in training new young employees, who currently receive 
the least amount of training. 

There are, in this formulation, three sets of indicators: 

FINAL OUTCOMES 

These are the final results we wish to achieve by improving the school to 
work transition. What is it, ultimately, that we want to improve? These 
measures are in terms of successful occupational and economic transition, 
because it is from “school to work” that is our focus. This in no way dimin- 



ishes other important outcomes of education, nor of other arrangements in 
society to provide for successful entry into adulthood. This is a set of 
indicators specific to this transition, not the general well being of Ameri- 
can youth. 



INTERMEDIATE OUTCOMES 

With the end result clearly in mind, what intermediate successes do we 
think will achieve these final outcomes? There is a lot of emphasis these 
days, and appropriately so, on “outcomes driven reform.” However, a 
statement of outcomes tells you nothing about how to achieve them. What 
has to happen for these outcomes to be realized? In bowling, there are 
markers (spots) a few feet down the lane. You decide which one you want 
the ball to roll over in order for it to hit the right place going into the pins. 

These markers are what I am calling intermediate outcomes. Select the 
wrong ones, and you won’t get there. These outcomes are not themselves 
the institutional changes you intend to bring about, but the results of 
those changes. 

SYSTEM OUTCOMES 

And what specific steps must we take during the transition to produce the 
intermediate outcomes we seek? These are indicators of whether the 
elements of a new system are in fact being put into place, and what por- 
tion of the total transition has been converted to a new system. Obviously, 
to do so, the elements of the new system must be specified. To have a 
relevant indicator system, consensus has to be reached on these elements. 

Of course, they could encompass some variation in approaches where 
consensus is not reached. As time passes it may be possible to see which 
approaches are associated with achieving the intermediate outcomes. 

However, an indicator system will not likely be a means of determining 
cause and effect; it is not a research design for a controlled experiment. In 
the bowling analogy, these program elements are the aim one takes at the 
markers. 

To put it simply, the system outcomes are the specific changes we want in institu- 
tional behaviors. These are designed to bring about the intermediate outcomes. The 
intermediate outcomes are expected to move us toward the final outcomes. Of course, 
the final outcomes will not result only from these intermediate outcomes, since other 
factors also affect their achievement. For example, a recession, or changes in 
demography, or the structure of production. 

We focus here on these changes in institutions because that is what we will set out 
to change through school-to— work legislation (state or federal), or through administra- 
tive action. Other factors will, however, play a large role in the behavior of the “final 
outcome” measures, and these are not tracked in the system described here. This larger 

(c 6 



set of factors that determine the transition are elaborated on in Volume 1 (of three 
volumes) of a publication called Between Two Worlds: Youth Transition From School to 
Work, by Paul Barton and Bryna Fraser, now 15— years-old. This volume postulated 
that there were three sets of factors: 1. Individual Circumstance; 2. Institutions: Qual- 
ity and Behavior; and 3. Changes in Society. Further, that these three sets of factors 
interact with each other. The system changes described in this paper have their origin 
in this earlier typology, although with considerable enlargement and, of course, updat- 
ing. Intermediate and Final Outcomes were not addressed at that time. 
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A. FINAL OUTCOMES 



1. Earlier Employment Establishment 

High school graduates have great difficulty establishing themselves in a way that 
enables them to become economically independent and create households and families. 
Their ability to do so has seriously deteriorated in the last two decades, absolutely and 
relative to their more highly educated peers. 



A. Indicator: 


Proportion of 18-20 year old graduates and 21-25 year old 
graduates, not enrolled in school, who are employed in the 
primary job market. 


Discussion: 


On the one hand, a youth labor market has developed, pay- 
ing around the minimum wage, having very high turnover, 
and having no (or minimal) hinge benefits. On the other 
hand, employers for adult-type jobs with career ladders and 
standard hinge benefits typically (but not always) do not 
hire those under the age of 22 to 25. Earlier employment 
establishment can happen only if earlier entry is gained to 
the primary market — one of the objectives of a new School 
to Work System. 


C onstruction: 


It would be constructed out of a combination of data on 
occupational and industry employment, earnings by occupa- 
tion, or fringe benefits by occupation, and age distribution 
by occupation. In some cases, arbitrary rules would be re- 
quired in deciding which occupations/industries would be 
counted as primary market. Such a composite of existing 
statistical series and rules for inclusion would result in an 
estimate to serve as an indicator, not a government 
“statistic.” 


Source: 


The BLS occupational survey, the payroll employment se- 
ries, the monthly CPS surveys, and (possibly for the most 
detail) the Decennial Census. I assume it can be constructed 
from existing data. 


B. Indicator: 


Percent earning enough to support a family that is a) above 
poverty and b) above “near poor ” by age 25. 


Discussion: 


The drop in real earnings among young high school gradu- 
ates, say age 21—25, threatens increased poverty and depen- 
dence, for them and the children they have (a reason for 
rising child poverty and near poverty). While our ambition is 



for young people to do better than escaping poverty and near 
poverty levels, one goal will be to turn around a trend toward 
impoverishment and dependency. 

Construction: Use of existing income statistics and official definitions of 

poor and near poor. Also could use dependency rates from 
AFDC and Food Stamp programs. However, the measure 
should be of those able to have a family and stay out of pov- 
erty, as well as of families already in poverty. At this age, we 
want to know how many have the ability to create families, 
and not live in poverty, as well as those who have already 
descended into poverty sunk into it. We are concerned with 
the ability to make this important life transition. 



Sources: Existing data: BLS, CPS, Census and HHS program data. 



2. Reversing the Relative Earnings Decline 

Indicator: The ratio of annual earnings and income of high school 

graduates and STW program completers* to the earnings of 
2-year and 4-year college graduates. 

Discussion: The increased economic difficulty of youth not pursuing 

college degrees is reflected in the deteriorating ratio of earn- 
ings and income of those with just high school educations 
(and those with some college, but without degrees, as well); 
they have been losing ground relatively over the last 20 
years. Quality School— to-Work programs should be expected 
to make successful completers more attractive to employers 
that pay higher wages. And they should spend less time 
unemployed. Other things being equal, success should show 
up in a halt in the relative earnings decline, and a reversal 
should begin. 

Of course, other things may not be equal and changing work 
and labor market structure could continue to produce these 
increasingly adverse ratios. It is a measure of how the whole 



’There is reference throughout to STW completer graduates. Definitions of what constitutes such a 
completer will have to evolve. It could be a high school graduate who went through an STW program, or 
a two-year college graduate. But regular high school programs could be evolving toward STW concepts 
and practices, without such formal designation, which has to do partially with funding sources. 
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Construction: 


of the society and economy is doing in terms of the welfare of 
this large segment of the youth population. In lower growth 
and stagnant economies, level of education certification tends 
to allocate scarce opportunities. 

From data in the Current Population Survey (Although the 
CPS measures years of school completed, not graduation. 
Since certifications of completion are important, it is some- 
thing we should know, particularly for those who have a 
degree after two years as opposed to just attending two 
years.) 


Sources: 


Above 



3. Rewarding Academic/Literacy/Skill Achievement 
in the Labor Market 



Indicator: 


The relative earnings and weeks worked of high school and 


Discussion: 


STW program graduates, age 17 to 21 or more, who have 
achieved at different levels and taken courses / training of 
different difficulty. Single age intervals should be used since 
age is a key factor in labor market penetration, and 21-years- 
old graduates will do better than those younger, at the same 
level of educational achievement. 

Until the labor market rewards differential achievement in 
school, taking more rigorous courses, and getting good skill 
training, efforts to create better School-to-Work programs 
will not succeed. Many studies, particularly those of John 
Bishop, have shown that for youth age 17 to 21 with high 
school diplomas or more, employers do not make such differ- 
entiations. If employers are not looking beyond the diploma, 
then changes in content and level of achievement will hardly 
be discovered. If there are no rewards, there are no economic 
incentives. 


Construction: 


A measure of course taking and achievement must be com- 
bined with data on labor market experience, preferably on a 
longitudinal basis. 


Source: 


In 1980, ASVAB was used together with a household survey, 
and is expected to be repeated for renorming. The best source 
is likely to be the NCES longitudinal surveys, such as NELS 
88. They can provide data from past surveys to establish 
trends, they deal with a single age cohort, and the longitudi- 
nal element can establish when differential accomplishment 



begins to get recognized. The periodic National Adult 
Literacy Studies could also be used. 



4. Increasing Economic Independence 



Indicator: 


The percent of youth* completing their formal educations, still 


Discussion: 


living at home with their families because they cannot afford 
to establish independent households (both from the perspec- 
tive of the youth and the perspective of the family). 

The measures in 1, above, relate to earnings necessary to 
establish economic independence. Another is the actual 
ability to establish households separate from the family, as it 
is perceived by the young adult and the family. Increasingly, 
there is the perception that youth are less able to move out 
and make it on their own. A measure of involuntary depen- 
dence on the family, even if subjective judgment, would help 
illuminate this matter. 


Construction: 


A question(s) in a household survey 


Source: 


Either the CPS or the Department of Education household 
survey 



5. The Perception of Well Being 



Indicator: 


Graduates who feel they are or are not succeeding in the labor 
market, do or do not feel in control of their lives, are or are not 
experiencing alienation. 


Discussion: 


The perception of well being results from both expectation 
and objective accomplishment. In a society and economy 
where more youth are likely not to do as well as their par- 
ents, there is likely to be more youth with perceptions of 
failure, which can lead to alienation. Objective measures 
(above) should be supplemented with subjective measures. 
Many would not consider this “hard” data. However, percep- 
tion is reality to an individual, and we must succeed on this 
front as well. 



There will be diff ering judgments as to the “age of economic independence” that we should strive for. 
This can be accommodated by measuring at ages 18, 21, and 25. 



Construction: From existing surveys if possible (I am not up to date on 



Source: 


ongoing surveys, such as Monitoring the Future, that might 
be used for this). 

Above 



B. INTERMEDIATE OUTCOMES* 

1. Increased Academic Achievement and Literacy 



Indicator: 


Proficiency in academic subjects and literacy. 


Discussion: 


Success in the transition to work will be aided, we believe, if 
students leaving high school and STW programs have greater 
academic and literacy achievement. 


Construction: 


Derived from the National Assessment of Educational 
Progres, the National Longitudinal Surveys, and the 
National Adult Literacy Study. 


Source: 


Above 



2. Increased Proportion of Secondary School Students in Bona 
Fide Work Based Learning Programs, Using the Worksite 



Indicator: 


The percentage of all 10- 12th grade students (as well as those 
in the postsecondary phase of STW programs who are in 
substantial worksite training / education programs), by how 
many hours are at the worksite. It would be desirable to 
deifferentiate four types of worksite:** 

a. Private enterprise 

b. Public agencies 

c. School-based enterprises 

d. Service learning opportunities 



•Wherever student populations are discussed, I assume collection of data by race/ethnicity, gender, and 
perhaps disability status. Programs for “out-of-school” youth are just different kinds of schools, and they 
would be included. 

**The preferred worksite is private employment. However, reality dictates that we contemplate other 
O mrces in order to get to scale. 



Discussion: 



Discussion: 


The belief behind STW programs is that using the worksite 




for training and experience, related to classroom work, will 
increase employment success, as well as stimulate educa- 
tional achievement. 


Construction: 


The Department of Education, to my knowledge, does not 
have a reporting system in place to build on for STW pro- 
grams. Reporting of Cooperative Education was abandoned 
several years ago. A state-based reporting system will be 
necessary. The types and intensities of the use of the 
worksite should be differentiated, through a generic classifi- 
cation, if possible, since terms like Co-op, Internship, and 
Youth Apprenticeship can have widely varying content and 
quality.* It is not just a matter of hours engaged at the 
worksite, but whether there is structured on-the-job train- 
ing and mentoring. This may be hard to capture statistically 
and on— site observation may be preferable. 


Source: 


State-level reports (with disaggregation below the state- 
level for state tracking purposes) to the Federal level. 



3. Increased Enrollment in Secondary School/Community College 
Articulated STW and Tech-Prep Programs** 



Indicator: 


Enrollments in secondary school in articulated programs that 
anticipate continuation at community colleges. 

Enrollments in community colleges of students who trans- 
ferred from the secondary education phases of articulated 
programs. 


Discussion: 


Such arrangements are considered essential to a complete 
STW program and improved transition for occupations re- 
quiring postsecondary degrees. Tracking enrollments will 
determine if efforts to develop and implement articulated 
programs is being translated into student engagement. We 
should also track policies of postsecondary institutions in 
whether they recognize STW credits for college admission. 



*A rough beginning at such a differentiation is contained in “A Memorandum on the Youth Transition,” 
by Paul Barton, in Improving the Transition From School to Work in the United States, American Youth 
Policy Forum, et al, 1993. 



**While not addressed here, we need some measure of completion rates. This is discussed some in the 
Appendix, Notes on “Performance Measures.” 



Construction: Standardized reporting from appropriate state agency or 

agencies, with reporting system worked out by the federal 
government with the states. Also, state by state reporting. It 
will be necessary to agree on rigorous definitions that iden- 
tify truly articulated programs. 

Source: New state and federal reporting, but building on presents 

reports under the Perkins Act. 



4. Increased Awarding of Skill Certification (and/or Employment 
Readiness Profiles Such as Worklink™ and Career Passports) 

Indicator: The proportion of graduates of high school and STW pro- 

grams that are awarded skill certifications: 

• recognized community wide 

• recognized statewide 

• recognized nationwide, by occupation or industry 

Discussion: A way of conveying employment readiness to employers, 

which is recognized by employers, is an essential feature of 
STW programs. There may be variation in the nature of the 
information, whether a cut— point approach to a specific 
occupational skill or a profile of work-related skills, behav- 
iors, and accomplishments. How many students are receiving 
such credentials/ profiles at the exit point of their educations 
is a measure of the effective reach of STW programs. 

Construction: New reporting system designed by DOL and DOE. However, 

it could also be approached through NELS ’88 type longitudi- 
nal surveys and transcript studies as such certifications get 
more institutionalized. 

Source: Same as above. 



5. Increases in the Proportion of High School Graduates Who Get 
Their Jobs with Assistance From the Institutions Involved in 
STW Programs (or at least elements of STW program) 

Indicator: The proportion of graduates entering employment who report 

getting first (and second) jobs through sources other than 
friends and relatives, such as teachers, school counselors, 
school and state employment services, employer organizations 
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and associations, the receipt of an occupational skill certifi- 
cate, contacts created through worksite learning programs, 
etc. 


Discussion: 


Due to the isolation of schools from the workplace, lack of 
assistance from agencies serving adults, such as the Employ- 
ment Service, the lack of school placement assistance, and 
the tendency of primary market employers not to hire gradu- 
ates until the early to mid 20s, most youth report getting 
their jobs through “friends and relatives.” Effectiveness can 
be gauged from seeing if more report graduates getting jobs 
in other ways. 


Construction: 


Expansion of the DOL October employment survey and the 
National Education Longitudinal survey such as NELS 88 
(and the latter at the state level at state option). 


Source: 


See above 



6. Increases in Employer Training of 18-24 year-old Employees 



Indicator: 


The proportion of employers of 18-24 year-old high school 


Discussion: 


graduates or students in STW programs who are providing 
entry-level or skill-improvement after hiring. 

Occupational preparation is inadequate both on the school 
side and the employer side. In many countries with which we 
compete, employers play a large role in training young work- 
ers. In the U.S., industry investment in training is generally 
low relative to our competitors. Furthermore, young workers 
receive the least amount of the total invested in training. 
Closing the school-to-work gap requires greater participa- 
tion by employers at the entry age, in their regular training 
programs. 


Construction: 


Data on entry-level and skill— improvement training is 
collected periodically (although not on a regular schedule) by 
the Bureau of Labor Statistics. The recent results and trends 
are summarized in a 1993 report, Training to Be Competitive, 
by the ETS Policy Information Center. 


Source: 


The occasional household survey, through the Current Popu- 
lation Survey, of workers and the training they receive from 
their employers. The survey is conducted by the Bureau of 
Labor Statistics. 



7. Changes in Parental Perception of the Viability of New School 
to Work Initiatives that are not Identified as a Traditional 
Academic Track/College Going Route 



Indicator: 


As new STW systems become widely available, the perceptions 
of parents about these systems and their desirability for their 
children.* 


Discussion: 


Parents generally fear what appears to them to be any form 
of tracking away from the college route. While STW pro- 
grams are designed to increase academic achievement, keep 
the college option open, and have a postsecondary compo- 
nent as part of an STW program or as an option, labels may 
make it hard to convince parents, and therefore, students. 
STW must succeed in terms of the parent perception of new 
systems as well as in the content of new systems. 


Construction: 


Could be questions added to household surveys, or parent 
surveys conducted by school systems. 


Source: 


Above 




C. SYSTEM OUTCOMES: 
Changes in Institutional Behaviors, 
Practices, and Services 



1. Growth in Worksite Training/Education Openings for 
Secondary (and Tech Prep) Students 



Indicator: 


Count of the number of schools whose students have opportu- 
nities, and the number of worksite slots involved, by a catego- 
rization (to be worked out) of a.) the extent of training time 
provided; b.) the level of skill certification involved upon 
completion; and c.) the relative involvement of employers and 
schools in direct occupational preparation. 



♦T his may be considered by some to be a soft measure, or a “sociological” measure. But these attitudes 
will make or break an STW effort, and should be gauged. 



Discussion: While this parallels Intermediate Outcome #2, we are here 

interested in measuring extent of program availability and 
type, rather than extent of penetration into the student body. 

Construction: Ideally, from reports filed at the school level. 

Source: School level reports or possibly, additions to the School and 

Staffing Survey, or transcript studies. 

2. Beyond Worksite Training/Education, Growth Generally in 

Secondary School and Tech-Prep Collaboration with Employers 

Indicator: A. Percent of secondary schools (and community colleges, 

regarding Tech Prep) having extensive involvement with 
employers and employer organizations 

B. Percent of employers having extensive involvement with 
secondary schools in efforts djrectly related to the transition to 
work (or further training and Tech Prep) 

Classify by nature of involvement, such as: 

• Communications with school personnel about students 
employed part-time, on their own 

• Assistance to schools in occupational counseling through 
providing mentors and advisors 

• Assistance to schools in expanding students* general knowl- 
edge of the workworld and the nature of various occupations 

• Adv ising on content of school— based occupational education 

Discussion: No new “system” for the school-to-work transition can come 

into being without close cooperation and collaboration be- 
tween the school and the employing community. Although 
the necessity for this was recognized in the first objective of 
National Goal 5 stating: 

“By the year 2000 — Every major American business will be 
involved in strengthening the connection between education 
and work,” 



no tracking system to monitor progress was put into place. 
What we are looking at here is help with the transition, not 



Discussion: While this parallels Intermediate Outcome #2, we are here 

interested in measuring extent of program availability and 
type, rather than extent of penetration into the student body. 



Construction: Ideally, from reports filed at the school level. 

Source: School level reports or possibly, additions to the School and 

Staffing Survey, or transcript studies. 



2. Beyond Worksite Training/Education, Growth Generally in 

Secondary School and Tech-Prep Collaboration with Employers 

Indicator: A. Percent of secondary schools (and community colleges, 

regarding Tech Prep) having extensive involvement with 
employers and employer organizations 

B. Percent of employers having extensive involvement with 
secondary schools in efforts directly related to the transition to 
work (or further training and Tech Prep) 

Classify by nature of involvement, such as: 

• Communications with school personnel about students 
employed part-time, on their own 

• Assistance to schools in occupational counseling through 
providing mentors and advisors 

• Assistance to schools in expanding students’ general knowl- 
edge of the workworld and the nature of various occupations 



• Advising on content of school-based occupational education 

Discussion: No new “system” for the school-to-work transition can come 

into being without close cooperation and collaboration be- 
tween the school and the employing community. Although 
the necessity for this was recognized in the first objective of 
National Goal 6 stating: 

“By the year 2000 — Every major American business will be 
involved in strengthening the connection between education 
and work,” 
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no tracking system to monitor progress was put into place. 
What we are looking at here is help with the transition, not 
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with the broad array of employer efforts to improve schools 
generally. 

Construction: This will require new reporting from schools and employers. 

It could be on a universe basis in a community; every 
employer, potentially, could be involved, as can every school 
(which is the goal of the Governors and the President, set in 
1989). At the state and national level it could be sample- 
based surveys. 

Source: If local surveys were guided by standardized instruments 

provided at the national level and adopted by the state, the 
data could be aggregated. However, given that this universal 
local reporting is a long way off, a sample survey would be 
required for national level reporting. The Goal 5 Task Force, 
working for the National Education Goals Panel, recom- 
mended surveys by the U.S. Department of Labor. Such 
surveys might be more successful if carried out with partici- 
pation of major employer organizations, such as the Confer- 
ence Board, the National Association of Manufacturers, the 
Chamber of Commerce, the National Alliance of Business, 
and the Business Roundtable. 



3. Extent of Integration of Academic and Occupational Education 
in Secondary Schools 

Indicators: A. The number of 

• courses taught jointly (or designed jointly) by academic and 
occupational education instructors 

• applied academic courses being taught such as CORD'S 
Principles of Technology 

B. The availability of instructional materials and teacher 
training opportunities for integrating academic and occupa- 
tional information. 

Discussion: Such integration is a key component of School-to-W ork 

Systems. The record keeping here is simply of the degree to 
which such integration is being achieved. There must be 
course offerings by subject matter area, and there must be 
suitable instructional materials developed and available for 
instruction, as well as arrangements for teacher training. 



Construction: Possibly, course availability could be obtained through modi- 

fication of transcript studies. The availability of instructional 
materials could be obtained through surveys made by the 
Council of Chief State School Officers in its indicator work. 
The SREB High Schools That Work Consortium could assist 
in specifying the indicators. 



4. Growth in Counseling/Guidance Time Available to Secondary 
School Students for Making Choices in School-to-Work 
Programs 

Indicator: The amount of counseling time provided to students overall 

(expressed as a ratio to students), the availability of computer- 
assisted guidance, and a distribution of counselor time along 
the lines of 

• class scheduling duties 

0 behavioral problems and psychological counseling 
0 selecting courses, majors, and tracks 

0 selecting colleges and arranging for college admissions tests 

0 providing occupational information for choosing careers 
and what is necessary to prepare for them 

0 providing information about available jobs 

Discussion: There has been no regular, or even periodic, reporting along 

the above fines, although the new School and Staffing Survey 
(SAS) does count the number of counselors. There was an ad 
hoc study in 1969 and also one in 1981*; and they disclosed 
that very little of available time was devoted to students 
going directly into the job market. 

Construction: Counselor ratios can be derived from the SAS surveys carried 

out by the National Center for Education Statistics. The time 
distribution could perhaps be obtained there also; counseling 
characteristics have not been treated on a par with teacher 
characteristics. 

Source: See above 




’Carried out by ETS under contract with the National Institute of Education. 
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5. The Provision of Placement Services to Graduating Seniors 



Indicator: 


The availability of job placement services to graduating 
students. 


Discussion: 


Organized job placement assistance is typically not available 
to graduating students, except on an ad hoc basis, commonly 
found in vocational schools. The Public Employment Service 
no longer provides this service (it used to, in about half of 
high schools), and systematic school-based services have 
seldom existed in the U.S. Failure to help graduating, work 
bound students is a peculiar characteristic of the U.S. sys- 
tem. 


Construction: 


This lies somewhere between the responsibilities of the U.S. 
Department of Labor and Education, as it does with their 
counterparts at the state and local level. Years ago, the 
Department of Labor quit asking State Employment Services 
for this information, probably because resource reductions 
had by and large eliminated the Cooperative School Program. 


Source: 


See above 



6. Establishment of Sequential Occupational Courses 



Indicator: 


In addition to counts of individual course offerings, the extent 
to which there are sequenced offerings, constituting a “major” 
in a particular occupational area. 


Discussion: 


The pattern in vocational education has been for students to 
elect a variety of courses in an unplanned way, so they end 
up without any entry-level skill in a particular occupation. 
In a School-to-Work system, a sequence of courses is estab- 
lished so there is progression to a marketable entry-level 
skill. The record keeping about offerings needs to show 
whether or not this is happening. 


Construction: 


Transcript studies designed specifically to reveal whether 
such sequencing is available. 


Source: 


Course reporting classifications developed by the Office of 
Adult and Vocational Education, linked to transcript studies 
carried out by NCES. 



7. Establishment of Skill Standards and Certification Systems 



Indicator: 


A count of occupations in which skill standards have been 
established, nationwide or in individual states, together with 
a count of certification systems available. 


Discussion: 


Having a credential recognized by employers is a critical 
element of a School-to-Work System. An indicator system 
would track the extent to which standards and certification 
are coming into existence. 


Construction: 


Possibly gathered through the new National Skill Standards 
Board from national and state industry associations and 
state departments of education and labor. 


Source: 


See above 



8. Growth in Secondary School/Community College Articulation 
Agreements 



Indicator: 


Number of articulation agreements that meet Tech-Prep 


Discussion: 


standards, categorized by the number of occupations in which 
there is a sequencing of courses over four years (2+2). 

Such agreements play a very important role in there being a 
postsecondary education option in School-to-Work Systems. 
An indicator system should include some means of tracking 
their availability. Rigorous definitions are needed. 


Construction: 


I am unfamiliar with the extent and nature of present report- 
ing for Tech-Prep programs and Perkins Act requirements. I 
assume that work has been done on this in the Office of 
Adult and Vocational Education. 


Source: 


See above 



9. Establishment of Agreements for Follow-up of Students for One 
Year (two?) After Graduation and Certification 



Indicator: 


The number of schools that provide follow-up, and nature of 
the follow-up. 
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Discussion: 


By and large, our students become invisible to the school 
system after graduation. The school doesn’t know what hap- 
pened to them (some have surveys shortly after graduation to 
determine if they were employed). This means that they no 
longer help in making a labor market adjustment. It also 
means they get no feedback that would enable them to adjust 
for the kinds of problems or deficiencies graduates have in 
gaining employment. 


Construction: 


The school would be the unit of reporting, or an STW system 
within a school if it is not school— wide. The objective though 
is for follow-up of all graduates. Perhaps they should remain 
on the school rolls as a responsibility for 12-24 months after 
graduation. 


Source: 


See above 



10. The Growth of Complete School-to-Work Transition Systems 



Indicator: 


The number of schools, or School-to-Work Systems located in 
schools, that include all (or most all?) the elements needed for 
a complete system. 


Discussion: 


While elements of systems are tracked in 1 through 9 above, 
and while each element is considered to be an improvement 
over the present, a complete system includes all the ele- 
ments. We need to know how many complete systems we 
have, at the school and part-of-school level. 


Construction: 


School level reporting aggregated to district, state, and 
national 


Source: 


See above 



D. T HE GEOGRAPHY OF INDICATORS 

There are several considerations in deciding on the geographical level of indicators. 
One is simply expense. No comprehensive indicator system will get off the ground if it 
has an astronomical price tag, and that’s what happens if certain kinds of indicators 
are planned for at the school, district, and state level in addition to the national level. 
However, school, district, and state level indicators become practical when they are 
designed to be built upon school reporting. 



A second consideration is whether it will be necessary to rely on household sam- 
pling, or whether administrative data can be used. Such sampling is typically expen- 
sive, and samples must be fairly large, even for small geographic areas; the number in 
the sample is as important as the size of the poulation. 

A third is the level at which decisions are made that can affect the matter being 
measured in the indicator. A school-based policy will likely best be tracked down to the 
school level. Those matters determined by economic policy at the national level. 

However, these are likely no clear principles that hold up for all the components of 
an indicator system. One reason is that the type of indicators vary, as in this presenta- 
tion of final outcomes, intermediate outcomes, and system outcomes. Also, the impor- 
tance of the target of measurement will affect the decision; a given target may be con- 
sidered so important that it has to be measured at the community level, even if it 
requires expensive sampling. So it comes down to examining one indicator at a time, 
and reaching consensus after grappling with factors like those above. 

I went through such as exercise, in a very rough way, and have presented the 
result in the following chart. It is the basis to start a discussion, but it is not the kind of 
thin g to be done by one person. For one thing, the indicators themselves are merely 
discussion starters; indicators will require considerable involvement from the policy 
and statistical communities, and we will have to face the reality test of cost and the 
availability of resources. No effort is made to document all the reasons in the author’s 
mind for making the choices reflected, but they do encompass the criteria described 
above, in a rough sort of way. 
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The Level of Reporting 





School 


Community 


State 


National 


Final Outcomes 


1A. 






Every 10 years. Through Census 
(or if large state in CPS) 


X 


B. 






X 


2. 






Every 10 years. Through 
Census (or if large state in CPS) 


X 


3. 






State option in longitudinal 
surveys 


X 


4. 








X 


5. 








X 


Intermediate Outcomes 


1 . 






State option in NAEP, NALS & 
NELS 


X 


2. 






State option in NAEP, NALS & 
NELS 


X 


3. 








X 


4. 


X 


X 


X 


X 


5. 


X 


X 


X 


X 


6. 


X 


X 


X 


X 


7. 






State option in NELS 


X 


System Outcomes 


1 . 


X 


X 


X 


X 


2. 


X 


X 


X 


X 


3. 


X 


X 


X 


X 


4. 


X 


X 


X 


X 


5. 


X 


X 


X 


X 


6. 






9 

• 


X 


7. 




9 

• 


X 


X 


8. 


X 


X 


X 


X 


9. 


X 


X 


X 


X 


10. 


X 


X 


X 


X 
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E. WHAT THE INDICATOR 
SYSTEM IS, AND IS NOT 



As broad as the reach of the indicator system described is, it is still purpose spe- 
cific, focused, and in some respects narrow. In using this document to aid a discussion 
of an indicator system, it may help to be clear about what it is and is not intended to 
be. 



It is a set of measures designed to track the putting into place (in whole geographi- 
cal areas) the elements of a system and whole systems (systems indicators), the inter- 
mediate outcomes of such a system, and the final outcomes (although these outcomes 
will be determined, as well, by larger happenings in the economy and society, as well as 
in the school— to— work system). It indicates change in the significant measures, and we 
should be able to judge whether or not the nation, or smaller geographical area, is 
making progress. 

It is: 

• Not a program development and implementation plan (although it makes 
assumptions about what the plan is, and should be adjusted to emerging reality) 

• Not an evaluation of a program (although, over time, som e judgments, not 
“findings”) can be made about whether payoffs occurred, although we will not 
know from indicators alone what elements produced the results we see. 

• Not a research design. It will not measure effectiveness of a system as com- 
pared to exist ing arrangements. The outcome sequence is not formally a causal 
model, and there are no controls. However, it embeds a set of presumptions 
about the elements of a system, what the shorter term results should be (inter- 
mediate outcomes), and what final results we want. 

• Not a set of performance measures and monitors for individual projects/funded 
programs. These deal with implementation targets, with conformance to pro- 
gram, regulatory, and legislative requirements, with resource utilization, and 
with expected outcomes set in advance for the project/program. Also, such mea- 
sures are at an individual project level, and not for all youth in entire geographi- 
cal areas (unless the program becomes universal). 

There is, however, likely to be some interplay between such project level moni- 
toring and a broad indicator system. 

• Not a record keeping system of the well being of all youth in general, but one 
focused on the transition of graduates to economic adulthood. 
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F. A PROCESS, NOT JUST A PLAN 



If this paper helps inform an initial discussion of the establishment of an indicator 
system, it would meet the expectations of the author. An indicator system will not be 
created by a plan put to paper by one individual. Rather, it will be created — assuming 
there is a strong desire to do so — out of a process involving a considerable number of 
people, coming together from different perspectives. There are the people who have 
created the demonstration programs and the new school-to-work legislation. There are 
the people in state— level government who will be implementing a school-to-work 
system. There are the people in state and federal administrative and statistical agen- 
cies who are fami liar with the existing reporting system and what its possibilities are 
for encompassing new measures. And there are the scholars who know particular areas 
covered by an indicator system in depth. 

While I have, in this paper, made an effort to cover the full span of a measurement 
system, I have done so largely with my own accumulated knowledge, which is quite 
incomplete and uneven. Some of the stabs I have taken at how to construct a measure 
will be wide of the mark. This paper did not evolve out of the process of involvement 
and consultation described above. 



* * * 



This effort springs from a strong conviction that we need an indicator system that 
permits monitoring the school-to-work transition as a whole. We need one that spans 
the education world and the workworld, mirrors the traversing by youth from one to 
the other, and illuminates the path(s) of entry into economic adulthood. What we have 
now largely parallels the separateness of education and work institutions. As Daniel 
Patrick Moynihan has said, “In America, we do what we measure.” We will need to 
break out of measuring only what, in the past, we have been accustomed to doing. To 
do so will require a collaborative effort between the Departments of Labor and Educa- 
tion and their counterparts in state government, and local institutions as well. 
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APPENDIX 

Notes on Performance Measures 



The School— to— Work Legislation that was proposed by the Clinton a dmi nistration 
and passed the House of Representatives in 1993 (and was awaiting action in the 
Senate when this was written) requires the establishment of “a system of performance 
measures for assessing state and local programs...” This is basically conceived as a set 
of accountability measures to assure quality, or to judge funded programs on the basis 
of achieving certain purposes of the legislation. As stated at the outset, this paper is 
addressing a comprehensive tracking system of the nation’s progress (or state’s... or a 
community’s ... or a school’s) in improving the school-to-work transition, and was not 
directly addressing these legislative requirements. 

One might view a set of performance measures for funded projects as a means of 
quality control in the establishment (by actions under this legislation) of a system. The 
proposed System Outcomes measures track the putting of elements of a system into 
place, not the quality of elements resulting from actions taken under the legislation. 
Beyond this difference, the indicator system proposed encompasses changes in institu- 
tional arrangements whatever their source, such as: 

— Actions under existing federal legislation, such as the Perkins Act, or laws adminis- 
tered by the Department of Labor for employment services and training 

— Actions taken by states on their own (or consortiums of states, such as the SREB a 
project) or communities on their own 

— Actions taken by individual postsecondary institutions and whole state sytems 

— Actions taken by individual schools and by employers in relation to those schools, or 
by employers acting alone in, for example, increasing their training of young entry 
level workers 



- Actions taken by industry associations in establishing skill standards, or otherwise 
spelling out entry requirements 



It is, of course, very important that the programs and projects under the new 
legislation be scrutinized through a set of measures, to judge project quality and help 
in decisions about continued funding of individual projects and programs. This seems 
to be the purpose of the performance measures described in the legislation. I do think 
there is benefit in starting from a broad perspective (whether or not the one described 
in this paper) of what we want to track to tell us whether we are creating a total sys- 
tem, and actually improving the transition, before specifying individual project mea- 
sures starting with specific legislative language. The broad view can perhaps focus 
down to the specific project. I read this legislative language after I wrote the paper; 
below are comments about these performance measures, one at a time. 
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(1) Progress in the implementation of state plans that include the basic program 
components and otherwise meet the requirements of Title I. 

Comment. This is a standard approach to administering a grant program and 
insuring that legislative and administrative requirements are adhered to. 

In this paper, the “basic program components” could be expected to be found in the 
System Outcomes indicators (although these may be spelled out in more detail than 
in the legislation). Individual projects and programs will have time tables for imple- 
mentation and start up; This is likely in the mode of a management-by-objectives 
type system. 

(2) Participation in school-to-work opportunities programs by employers, schools, 
and students. 

Comments. It is not so clear what this means on a project basis, in specific. By the 
legislation, the funded programs have to show participation by employers and 
/ schools. If there are not students participating, there is certainly no program. This 
sounds like a matter of compliance with the terms and conditions for receiving 
funding, and continuing to meet the specifications for having at least the bare ele- 
ments of a school-to-work program. 

If it is me an t that such participation should grow throughout the nation, or a state, 
or a community — stimulated by this legislation, that is, increasing participation 
rates — then this is addressed in Intermediate Outcomes fairly extensively. 

(3) Progress in developing and implementing strategies for addressing the needs of 
in— school and out— of— school, at— risk youth. 

Comments. This seem to be a lot like the first: are the recipients of funds doing 
what they agreed to do... the conditions for receiving grants in the first place. It is a 
bit confusing though, for, in the case of in-school youth at risk, it is the STW pro- 
gram itself that is the strategy. It would be unfortunate if sound efforts to better 
meet the needs of these youth (who may not seem “at risk ” if they are enrolled in STW 
programs) would be undercut by highly visible performance / accountability measures 
that belie the objective of having this effort as perceived as being for all youth. An- 
other attempt at labeling the clients will fail both these clients and the student body 
as a whole. 

As for “out-of-school” programs, if they meet all the standards set for STW pro- 
grams, I would treat them the same as in-school programs, in an indicator system. 
If they result in GEDs, there should be articulation agreements with Tech Prep 
programs (as an option), just as high schools do. These programs also need to inte- 
grate strong academics, and lead to skill certification. Otherwise, they should not be 
receiving STW funding. 
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(4) Student outcomes, including — 

(A) Academic learning gains 

Comments: Addressed in Intermediate Outcomes (1). 

(B) Staying in school and attaining a school diploma, skill certificate, and college 
degree 

Comments: Credentials recognized by employers is 

addressed. Reducing the dropout rate probably should have been addressed under 
Intermediate Outcomes; dropout rate measures, however, are being improved at the 
state and national level by the National Education Goals Panel, the Council of Chief 
State School Officers, and the National Center for Education Statistics. These sta- 
tistics will exist and can be used in an indicator report. 

The matter of dropout rates for individual funded programs is somewhat different. 

If they are comparative, then compared to what? If a whole school has STW, then it 
can be compared to the past. If compared to a standard then how to set the stan- 
dard? The best approach, would, I believe, be to demonstrate continuous improve- 
ment, and narrowing of differential rates among sub- populations within a school 
(or a program). 

The inclusion of “college degree” requires some care in developing this measure and 
what its role is to be. College is an option for individuals in STW programs and 
should remain so, even as we would like more students in the programs articulated 
with 2 year colleges. We are, I believe, trying to develop options , options that are 
limited now because college is really the only viable option at present. As an option, 
this is addressed in both Intermediate and System Outcomes. 

(C) Placement and retention in further education and training, particularly in the 
student’s career major; 

Comments: It is an important objective to provide the quality academics and train- 
ing that keep education options open, and to encourage all young people to develop 
to their full potential. But whether they do or not, in high percentages, is not, in my 
judgment, a make or break matter for a funded project or program. 

One of the great problems of the American education system, I believe, was (and is) 
characterized by Rupert Evans about 25 years ago. Each stage of the system, he 
said, is designed to prepare students for the next level of education. Not for exiting 
from the system. I think the STW effort is one means to correct this. We want its 
graduates to be prepared to exit into the workforce, whether or not they choose to 
continue. That’s why we want them to have skill certificates recognized by employers. 
This includes, of course, STW programs in which the final year, or two years, is in a 
postsecondary institution. 
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It is also not apparent why this measure is placed before the employment measure 
(below). 

(D) Job placement, retention, and earnings, particularly in the student’s career 
major 

Comments: I have dealt with these outcomes under Final Outcomes. But for project 
accountability, there is the further question of standards. How good is good enough? 
Success rates compared to what? How quick should success be achieved? There has 
been a lot of experience with this in JTPA, including recognition of varying labor 
market conditions, that will likely be drawn upon. It is an important measure cen- 
tral to the purposes of the STW program. 

However, while these are bottom line outcomes, we need to recognize the continued 
development that takes place in the transition to economic adulthood, or what 
Marcia Freedman called, years ago, the process of work establishment, the title of a 
book she wrote. It does not frequently happen on a fixed schedule, even for four year 
college graduates. They do not necessarily end up “in their majors,” or start out that 
way either. The question is whether their employment trajectory from say 18 to 24 
is greatly improved, in terms of participation in the primary market, or in rise in 
relative earnings. It is harder to measure a trajectory, of course, but it would be an 
oversimplication just to measure whether they are “employed full time in their 
major three months after they graduate.” 

(E) The extent to which the program has met the needs of employers 

Comments: This can be learned by following up every graduate for a substantial 
period of time, as proposed. Also, surveys of employer participation will be helpful. 
Of course, employers have a responsibility for training entry level employees — a 
responsibility fulfilled much better in countries with which we compete. They are 
serving themselves by being a party to occupational preparation. A performance 
measure should not send the message that public agencies are solely responsible. 
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